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INTRODUCTION 


Psychologically-based  treatments  and  cognitive  behavioral  therapy  (CBT)  interventions  have 
been  shown  to  be  effective  in  alleviating  symptoms  of  Post-Traumatic  Stress  Disorder  (PTSD) 
and  related  psychological  health  difficulties  in  Veterans  and  military  personnel  who  suffer  from 
these  problems.  To  meet  the  increased  service  needs  of  Veterans  with  PTSD,  new  training 
methods  need  to  be  developed  which  are:  1)  evidence-based,  and  2)  effective  in  modifying  and 
sustaining  changes  in  provider  behavior.  Methods  of  training/implementation  must  also  be 
scalable,  and  feasible  for  delivery  to  large  numbers  of  providers  in  cost-effective  ways. 
Internet-based  training  is  a  promising  new  approach  for  meeting  this  need,  but  has  received 
little  systematic  evaluation  to  date.  Noting  the  urgency  and  high  priority  of  this  issue,  Fairburn 
and  Cooper  (2011)  have  advocated  strongly  for  the  development  of  novel,  internet-based 
training  methods  and  innovative  research  designs  to  test  the  effectiveness  of  these  new  training 
methods.  Our  current  program  of  research  is  aimed  to  address  these  needs. 

The  broad  objective  of  our  research  is  to  design,  implement  and  evaluate  scalable  and  cost- 
effective  new  methods  for  training  of  mental  health  clinicians  providing  treatment  services  to 
veterans  with  PTSD.  The  randomized  controlled  trial  (RCT)  design  is  briefly  as  follows:  eligible 
clinicians  in  the  community  and  VPIA  will  be  randomly  assigned  in  equal  numbers  to  three 
parallel  intervention  condition:  a)  Web-based  training  plus  web-centered  supervision;  b)  Web- 
based  training  alone;  and  c)  Training-as-usual  control  group.  An  equal  number  of  clinician 
trainees  from  VPIA  (N=219)  and  the  community  (N=219)  will  be  recruited  and  enrolled  in  the 
study  over  an  18-month  period  according  to  a  randomized,  stratified  24-week  design. 

Comprehensive  assessments  will  be  performed  at  baseline  (TO),  completion  of  training  (T1), 
and  at  3  month  follow-up  (T2).  Participants  randomized  to  the  consultation  condition  will  be 
exposed  to  a  newly  developed  web-centered  form  of  learning  consultation.  Measures  of 
compliance  and  completion  will  assess  adherence  to  protocol.  Training  effectiveness  will  be 
evaluated  by  means  of  a  combination  of  objective  (SPE)  and  self-report  measures. 

The  primary  and  secondary  aims  of  the  study  are  as  follows: 

Primary  Aim:  To  compare  an  enhanced,  internet-based  training  intervention  combined  with 
novel  web-centered  supervision,  internet-based  training  intervention  without  web-centered 
supervision  and  a  wait-list  control  with  regard  to  improvements  in  two  CBT-based  skill  areas 
(behavioral  task  assignment  and  chain  analysis).  We  hypothesize  that  enhanced,  internet-based 
training  in  conjunction  with  web-centered  supervision  will  result  in  superior  skills  acquisition 
compared  to  internet  training  alone  and  that  internet  training  alone  will  result  in  superior  CBT 
skills  than  wait-list  control. 

Secondary  Aim  #1:  To  compare  improvements  in  knowledge  and  attitudes  following  internet- 
based  training  with  or  without  web-centered  supervision  and  the  control.  We  hypothesize  that 
web-centered  supervision  will  lead  to  greater  improvements  in  CBT  knowledge  and  perceived 
self-efficacy  compared  to  internet-based  training  without  supervision  or  a  written  training-only 
condition.  We  hypothesize  similarly  that  internet-based  training  will  be  associated  with  improved 
outcomes  in  CBT  knowledge  and  attitudes  compared  to  a  written  training-only  condition. 

Secondary  Aim  #2:  To  compare  improvements  in  skills  acquisition  in  knowledge  and  attitudes 
following  training  in  clinicians  recruited  from  VFIA  mental  health  treatment  settings  compared  to 
those  providing  services  in  civilian  community-based  clinics.  We  hypothesize  that  comparable 
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improvements  will  be  achieved  in  the  trainees  from  civilian  community-based  clinics  compared 
to  clinicians  recruited  from  VHA  centers. 

Secondary  Aim  #3:  To  assess  the  relative  efficiency  of  training,  as  measured  by  total  time 
required  for  training  in  each  condition,  in  addition  to  self-reported  level  of  burden  for  clinicians. 
We  hypothesize  that  internet-based  training  with  or  without  web-centered  supervision  will  be 
associated  with  increased  time  investment  and  burden  relative  to  training-as-usual,  but  that 
absolute  levels  of  burden  will  be  low  in  the  web  training  conditions. 

This  study  will  be  the  first  of  its  kind  to  systematically  compare  web-based  training  interventions 
across  treatment  settings  and  provider  groups  (VHA  vs  Non-VHA).The  study  will  also  be  unique 
in:  1)  developing  and  testing  of  new  web-enhanced  training  modules  and  a  novel  web-centered 
supervision  model  recently  proposed  by  Fairburn  &  Cooper  (2011);  2)  development  and 
implementation  of  a  new  patient-reported  measure  of  clinician  skill  and  competency;  and  3) 
assessment  of  post-training  maintenance  of  skills  beyond  the  training  period.  Our  focus  on 
broad-based,  generic  CBT  skills  rather  than  more  narrowly  focused  protocol-based  skills  is 
another  innovative  aspect  of  our  proposed  study.  Finally,  the  use  of  standardized  patient 
methodology  for  assessing  outcomes  of  training,  and  planned  comparisons  with  self-report  and 
knowledge-based  assessment,  is  another  novel  feature  of  our  proposed  study. 

If  successful,  the  study  will  promote  a  better  standard  of  care  for  psychological  health  of 
Veterans  and  their  families  by  evaluating  technical  feasibility  of  two  training  models  in  evidence- 
based  skills  for  PTSD  treatment  providers  and  measuring  their  outcomes  and  effectiveness.  If 
successful,  the  study  will  provide  experimental  support  for  broad  implementation  of  these 
enhanced  new  training  methods  across  a  variety  of  treatment  settings. 

KEYWORDS 

Behavioral  Task  Assignment  (BTA) 

Chain  Analysis  (CA) 

Cognitive  behavioral  therapy  (CBT) 

Post-Traumatic  Stress  Disorder  (PTSD) 

Randomized  controlled  trial  (RCT) 

Standardized  Patient  (SP) 

ACCOMPLISHMENTS 

This  section  of  the  report  will  describe  the  research  accomplishments  associated  with  each  task 
outlined  in  the  approved  Statement  of  Work  (SOW). 

Year  1: 

T  Develop  Web-based  Training  Materials 

a.  Develop  and  pre-test  CBT  instructional  modules  and  materials 

i.  During  the  first  year,  the  content  development  modules  were  desgined, 
developed  and  completed.  Module  content  for  the  video  clip  development 
was  finalized,  including  filming  and  final  cut  review. 

b.  Develop  case  material  and  demonstrations 

i.  During  the  first  year  of  this  project,  the  team  developed  multiple  case 
portrayals  for  use  in  the  video  simulation  interviews. 

c.  Prepare  web-based  supervision  manual  and  materials 
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i.  During  the  first  year  of  this  project,  the  team  created  and  finalized  a 
manual  and  materials  for  the  web-based  supervision. 

d.  Training  content  complete,  reviewed  by  CBT  expert  consultants 

i.  A  team  of  carefully  selected  expert  content  consultants,  including 
Christopher  Fairburn,  MD,  Amy  Naugle,  Ph.D.,  Gareth  Holmen,  Ph.D., 
and  Brett  T.  Litz,  Ph.D,  were  invited  to  serve  as  special  consultants  to  the 
project  based  upon  their  extensive  experience  and  knowledge  in  these 
content  areas. 

e.  Pilot  test  all  study  procedures  and  materials  (prior  to  programming) 

i.  Pilot  testing  of  the  knowledge  items  was  initiated  with  >  50  completed 
surveys  collected  and  both  qualitative  and  quantitative  data  collected.  A 
combination  of  blueprinting  and  concept  mapping  approaches  were  used 
to  identify  key  concepts  and  to  generate  specific  knowledge  assessment 
items.  The  assessment  items  were  being  pre-tested  according  to  our 
study  protocol  and  final  items  were  later  available  at  randomization. 

2.  Develop  and  Finalize  Study  Protocol  and  Measures 

a.  Eligibility  criteria,  exclusion  criteria,  screening  protocol 

i.  During  the  first  year  of  this  project,  the  team  finalized  the  eligibility  criteria, 
exclusion  criteria  and  screening  protocol. 

b.  Sample  frame,  web  contact  methods,  email  lists  identified 

i.  During  the  first  year  of  the  project,  the  team  identified  potential 
participants  as  well  as  identified  exisiting  email  lists  of  participants 
through  collaboration  with  community  based  groups. 

1 .  Email  lists  included:  VPIA  mental  health  clinicians  that  have 

participated  in  the  “clinical  training  program”  at  the  National  Center 
for  PTSD  in  the  past,  VPIA  mental  health  clinicians  who  have  not 
participated  in  the  “clinical  training  program”  at  the  NCPTSD,  team 
leaders  at  the  Vet  Centers  in  the  United  States,  and  a  registry  of 
community  practitioners  who  have  agreed  to  volunteering  pro 
bono  to  serve  the  needs  of  veterans  (Give  an  Flour).  A  large 
number  (6800)  of  community  clinicians  participate  in  this  registry 
and  our  study  has  arranged  access  to  this  recruitment  pool. 

ii.  Postcards  were  created  to  be  disseminated  at  conferences  and  other 
mental  health  events  to  enhance  recruitment  activities  in  addition  to  the 
pre-defined  email  lists. 

c.  Standardized  Patient  Interviews  scripted  and  pre-tested 

i.  A  standardized  patient  interview  script  was  created  for  assessing  clinicial 
skills  prior  to  and  following  the  training.  It  was  pre-tested  during  year  1 . 
Actors  and  interviewers  were  identified  for  the  study. 

d.  Research  measures  and  instrumments  developed  and  pre-tested 

i.  Knowledge  and  attitude  questionnaires  were  developed  for  the  study  and 
were  underwent  pre-testing  during  year  1.  Other  measures  were  finalized 
based  on  findings  from  the  first  study  (Ruzek  et  al.,  2012). 

e.  Consent  form  drafted,  human  subjects  protocol  finalized 

i.  During  the  first  year  of  this  project,  the  web  based  consent  form  was 
created.  The  consent  form  and  protocol  were  approved  by  both  NERI  and 
the  Stanford  IRBs.  The  terms  of  the  study  were  clearly  defined  with  an 
“accept”  or  “decline”  participation  button.  All  consent  activities  occured 
through  a  secure  web  site. 

f.  NERI/VHA  IRB  approvals  and  USAMRMC  HRPO  human  subject  protocol 
approval 
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i.  The  Protocol,  Recruitment  Materials,  Informed  Consent  documents, 
applicable  applications  were  approved  by  the  New  England  Research 
Institutes,  Inc  (NERI)  IRB  on  May  6,  2013.  Revised  Consent  forms 
(version  4)  were  approved  by  the  NERI  IRB  on  June  20,  2013  to  comply 
with  the  DoD  requirements.  The  protocol  was  reviewed  by  the  US  Army 
Medical  Research  and  Materiel  Command  (USAMRMC),  Office  of 
Research  Protections  (ORP),  Human  Research  Protection  Office  (HRPO) 
and  found  to  comply  with  applicable  DOD,  US  Army,  and  USAMRMC 
human  subjects  protection  requirements.  This  approval  was  dated  July  2, 
2013. 

3.  Develop,  Pre-Test  and  Finalize  Web  Site  and  Instructional  Program 

a.  I /HA  web  host  programmers  provide  specification  and  guidance  to  web 
programmers  and  database  programmers 

i.  The  team  collaborated  with  the  VHA  NCPTSD  to  determine  integration 
requirements  for  ultimately  hosting  the  of  the  programmed  in  the 
construction  of  our  web-based  training  program  on  an  NCPTSD  server. 
The  site  is  fully  tested  and  compliant  with  WCAG  and  508  compliance 
requirements. 

b.  PTSD  training  material  completed.  All  web  training  modules  have  been 
completed  and  wre  being  beta  tested. 

c.  Web  programming  specifications  completed 

i.  During  the  first  year  of  this  project,  the  web  programming  specifications 
were  identified,  content  was  finalized  and  web  programming  activities 
were  initiated. 

4.  Develop  Data  Management  System 

a.  Flow  chart  all  study  steps,  web  data  collection  and  database  requirements 

i.  During  the  first  year  of  this  project,  a  design  was  created  to  show  the 
steps  in  data  collection  to  plan  for  programming  of  the  DM  system. 

b.  Develop  web-based  data  forms  for  all  research  measures 

i.  During  the  first  year  of  this  project,  the  team  started  the  creation  of  web- 
based  data  forms. 

c.  Program  the  database  including  project  data  entry  forms 

i.  During  this  period,  the  team  conducted  initial  programming  activities  of 
forms  in  preparation  for  the  pilot. 

d.  Program  web  usage  statistics  reports 

i.  An  analysis  plan  was  developed  for  selection  and  validation  of  knowledge 
items. 

e.  Program  automatic  email  reminders/interaction  with  web  participants 

i.  During  the  first  year  of  this  project,  the  email  reminder  process  had  been 
reviewed  and  was  an  ongoing  part  of  the  data  management  activities. 

5.  Prepare  Standardized  Patient  Rating  Protocol 

a.  Develop  and  Pre-test  Standardized  Patient  (SP)  Hating  Guide 

i.  Standardized  patient  case  materials  were  developed  and  were  in  a  pre¬ 
testing  phase  during  year  1. 

ii.  New  rating  scales  had  been  developed  and  are  were  being  pre-tested 
during  year  1 . 

b.  Training  proceedures  were  developed  and  pre-tested. 


Year  2: 


T  Develop  and  Finalize  Study  Protocol  and  Measures 
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a.  Web  programming  specifications  were  completed 

b.  Web  modules  were  programmed 

c.  Beta  testing  of  web  program  occurred 

2.  Develop  Data  Management  System 

a.  Programming  of  the  the  database  including  project  data  entry  forms  was 
completed 

b.  Programming  of  reports  was  completed 

c.  Beta  testing  of  all  programmed  pieces  and  interactivty  was  conducted 

d .  Ongoing  monitoring  of  study  process 

i.  Recruitment  started  on  3/5/2014 
i  i .  FPI  was  on  3/7/20 1 4 

iii.  During  the  second  year  of  this  project,  the  team  actively  monitored  data 
collection,  data  management,  and  the  study  process.  We  developed 
weekly  reports  for  active  tracking  of  recruitment  and  study  processes,  and 
met  bi-weekly  to  discuss  progress. 

3.  Prepare  Standardized  Patient  Rating  Protocol 

a.  Develop  and  Pre  Test  Standardized  Patient  (SP)  Hating  Guide 

i.  During  the  second  year  of  this  project,  the  team  developed  the  SP  Rating 
Guide.  Rating  scales  were  developed,  and  underwent  careful  review  for 
inter-rater  reliability  and  finalization  of  all  rating  items,  to  assess 
adherence  and  competence  for  each  training  module. 

b.  External  Review  of  the  SP  Rating  Guide  and  Methods 

i.  A  metrics  committee  was  formed,  comprised  of  key  study  staff  and 
external  advisor  Dr.  Zafra  Cooper  (Oxford). 

ii.  Our  team  developed  the  rating  guide  and  methods  were  reviewed  by  the 
metrics  committee,  and  Dr.  Zafra  Cooper  (Oxford)  assisted  with  final 
development  and  review. 

c.  Training  of  Independent  Study  Raters  to  Concordance 

i.  The  team  hired  and  trained  four  study  raters. 

4.  Develop  Data  Collection,  Intervention  and  Standardized  Patient  Interviews 

a.  Web  screening  of  potential  clinician  applicants  occurred. 

b.  SP  baseline  interviews  with  eligible  clinicians  occurred. 

c.  Automated  random  assignment  of  participants  with  Completed  SP  Interview  by 
Web  Program  to  Web-Course,  Web-Course  +  Web-  Based  Supervision,  or 
Control 

5.  Analysis  and  Evaluation 

a.  Monitor  Data  Collection  Rates  and  Data  Quality 

i.  The  team  collaborated  with  the  VHA  NCPTSD  to  monitor  the  collection 
rates  and  data  quality  on  a  weekly  basis. 

b.  Create  Interim  and  Final  Analytical  Data  Sets 

i.  During  the  second  year  of  this  project,  the  initiation  of  the  baseline  data 
sets  began. 


Year  3: 
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Y  Develop  Data  Management  System 

a.  Ongoing  monitoring  of  study  process 

i.  Between  years  2  and  3  of  this  project,  the  team  actively  monitored  data 
collection,  data  management,  and  the  study  process.  Bi-weekly  reports 
were  developed  for  active  tracking  of  recruitment  and  study  processes, 
and  met  bi-weekly  to  discuss  progress.  Study  enrollment  was  completed 
in  April  2015  with  a  total  of  420  participants  (VA  =  209,  Community  = 

211),  exceeding  our  targeted  recruitment  goal  of  414  participants  (207  per 
sector) 

2.  Prepare  Standardized  Patient  Rating  Protocol 

a.  Develop  and  Pre  Test  Standardized  Patient  (SP)  Hating  Guide 

i.  The  team  developed  the  SP  Rating  Guide  in  year  2.  During  year  3,  rating 
scales  were  developed  and  finalized. 

b.  External  review  of  SP  rating  guide  and  methods 

i.  During  year  3,  raters  utilized  the  finalized  rating  forms  to  rate  the  SPs 
from  BTA  and  CA.  These  ratings  underwent  continued  analyses  for  inter¬ 
rater  reliability. 

c.  Training  of  Independent  Study  Haters  to  Concordance 

i.  The  team  hired  and  trained  study  raters  for  BTA  and  CA.  Blinded 
independent  ratings  and  reliability  assessents  began  in  Q2  of  Year  2. 

3.  Data  Collection,  Intervention  and  Standardized  Patient  Interviews 

a.  Schedule  and  Carry  Out  Post  Training,  Follow  Up  SP  Interviews 

i.  This  was  an  ongoing  occurrence  into  year  3. 

4.  Analysis  and  Evaluation 

a.  Monitor  Data  Collection  Hates  and  Data  Quality 

i.  The  team  collaborated  with  the  VHA  NCPTSD  to  monitor  the  collection 
rates  and  data  quality  on  a  weekly  basis. 

b.  Create  Interim  and  Final  Analytical  Data  Sets 

i.  During  the  second  year  of  this  project,  the  initiation  of  the  data  sets  was 
scheduled  to  begin.  This  task  continued  into  year  4.  Final  datasets  will  be 
completed  in  Year  5  (following  approval  of  extension  without  funding). 


Year  4: 

T  Prepare  Standardized  Patient  Rating  Protocol 

a.  Training  of  Independent  Study  Haters  to  Concordance 
i.  All  ratings  were  completed  in  year  4. 

2.  Data  Collection,  Intervention  and  Standardized  Patient  Interviews 

a.  Data  collection  was  completed  on  January  7th,  201 6. 

b.  A  complete  dataset  was  downloaded  and  saved  to  a  secure  server. 

c.  SP  ratings  rubric  was  finalized. 

3.  Analysis  and  Evaluation 

a.  Monitor  Data  Collection  Hates  and  Data  Quality 

i.  The  team  collaborated  with  the  VHA  NCPTSD  to  monitor  the  collection 
rates  and  data  quality  on  a  weekly  basis. 

ii.  Data  cleaning  and  recoding  was  performed. 

b.  Create  Interim  and  Final  Analytical  Data  Sets 

i.  During  the  second  year  of  this  project,  the  initiation  of  the  data  sets 
began.  This  task  continued  into  year  4  and  final  datasets  will  be  comleted 
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in  year  5  (following  approval  of  the  extension  without  funding).  A 
statistical  analysis  plan  was  developed. 

c.  Peform  all  analyses  according  to  specification,  share  output  and  findings  with  all 
investigators 

i.  Analysis/proposal  plans  for  the  baseline,  main  results,  and  web  usage 
papers  have  been  created  and  reviewed  by  the  investigators. 

d.  Author  and  co-author  evaluation  findings 

i.  All  three  papers  are  in  the  process  of  being  written. 

e.  Convene  advisory  meeting 

i.  All  advisory  personnel  have  been  kept  informed  of  study  progress 
throughout  the  study  duration. 

ii.  A  final  meeting  will  be  held  in  year  5. 

4.  Other  Items 

a.  Conversion  and  long  term  hosting  of  the  study  website 

i.  NERI  has  completed  conversion  of  the  web-based  training  program  from 
a  Content  Management  System  (CMS)  to  HTML,  incorporating  all  training 
modules  developed  by  NERI  and  the  VA  (Behavioral  Task  Assignment, 
Goal  Setting,  Motivational  Interviewing,  and  supplemental  videos  for 
Chain  Analysis),  and  removing  the  login  required  during  the  study  period 
to  allow  for  open  access.  NERI  has  provided  final  HTML  files  to  the  VA 
for  long-term  hosting.  NERI  remains  available  to  discuss  questions 
regarding  hosting  the  HTML  site  on  the  VA/EES  server. 

ii.  NERI  will  continue  to  host  the  study  site  until  September,  2018,  to  allow 
the  VA  adequate  time  to  identify  a  long-term  hosting  solution. 

b.  CE/CME  accredidation 

i.  All  CE/CME  certificates  have  been  issued  for  completion  of  the  training. 
Four  types  of  accreditation  were  offered:  Social  Work,  Nursing,  CME,  and 
Psychology.  Participants  received  5  credit  hours  for  completion  of  this 
training.  All  participants  who  completed  the  follow-up  assessment  were 
eligible  to  receive  certification.  In  order  to  receive  a  certificate, 
participants  were  required  to  complete  a  CE  evaluation  (administered  at 
post-test,  and  re-sent  to  participants  who  did  not  complete  it  during  the 
study),  and  pass  the  knowledge  assessment  (at  either  post-test  or  follow¬ 
up)  with  a  score  of  70%  or  higher  in  order  to  earn  the  certificate.  If  a 
participant  did  not  pass  the  knowledge  assessment  during  the  study,  they 
were  sent  a  re-test  of  the  knowledge  questions  following  completion  of 
their  participation.  Participants  were  allowed  to  take  as  many  re-tests  as 
necessary  in  order  to  pass  the  knowledge  assessment.  A  number  of 
participants  opted  out  of  receiving  a  CE  certificate,  or  did  not  respond  to 
our  requests  to  complete  a  re-test.  In  total,  139  continuing  education 
certificates  were  issued. 

c.  Presentations  of  prelimnary  findings 

i.  Preliminary  findings  for  baseline  and  main  results  data  were  presented  at 
the  Military  Health  System  Research  Symposium  (MHSRS)  in  August 
2016. 

ii.  Preliminary  findings  for  web  usage  data  were  presented  at  the  American 
Psychological  Association  (APA)  2016  Convention  in  August  2016. 

iii.  A  Symposium  will  be  presented  at  the  International  Society  for  Traumatic 
Stress  Studies  (ISTSS)  2016  Annual  Meeting  in  November  2016, 
covering  consultation,  standardized  patient  methodology,  and  web  usage 
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results. 


Final  recruitment  information  is  as  follows  (as  of  1/7/2016): 

Invitation  Statistics 

34,495  potential  participants  were  invited  to  join  the  study 

Registration  Statistics 

1609  registrant 

Screener  Statistics 

Out  of  1 609  registrants.. . 

998  participants  have  screened  eligible 
68  participants  have  screened  ineligible 
543  participants  have  not  screened 

Consent  Statistics 

Out  of  998  participants  screened  eligible... 

856  participants  have  consented 
64  participants  have  not  consented 
78  participants  have  not  completed  the  consent 

Pre-test  Statistics 

Out  of  856  participants  consented... 

613  participants  have  completed  the  pre-test 

Standardized  Patient  1  Statistics 

Out  of  613  participants  who  completed  the  pre-test. . . 

420  participants  have  completed  the  SP1 

Randomization  Statistics 

Out  of  4 20  participants  who  completed  the  SP1 ... 

420  participants  have  been  randomized 

66  participants  have  been  randomized  but  have  decided  to  not  partake  in  the  study  (early 
termination) 

Post- test/S P2  Statistics 

Out  of  4 20  participants  who  were  randomized... 

273  participants  have  completed  the  SP2 
241  participants  have  completed  the  post-test 

Follow-up  Statistics 

Out  of  the  420  participants  who  have  been  randomized  ... 

224  participants  have  completed  the  Follow-up 

Standardized  Patient  3  Statistics 

Out  of  420  participants  who  have  been  randomized. . . 

244  participants  have  completed  the  SP3 
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KEY  RESEARCH  ACCOMPLISHMENTS 


•  Web-based  and  written  training  materials  were  finalized 

•  Video  elements  were  implemented  into  the  training  website 

•  Training  website  was  launched,  to  include  Behavioral  Task  Assignment  &  Chain 
Analysis  and  Case  Formulation 

•  CE  accreditation  was  obtained  for  Psychology,  Social  Work,  Nursing,  and  CME 

•  Standardized  Patient  Interview  scripts  were  finalized,  and  actors  were  hired  and  trained 

•  Pre-test,  post-test,  and  follow-up  questionnaires  were  developed,  finalized,  and 
programmed 

•  All  data  collection  forms  were  finalized,  programmed,  and  tested  in  the  data 
management  system 

•  Reports  to  review  ongoing  statuses  of  all  forms  and  subject  progress  were  developed 
and  finalized 

•  Subject  recruitment  methods  were  finalized  and  recruitment  was  started 

•  Subject  management  procedures  for  the  entire  study  flow,  including  all  email  and  phone 
communications  were  developed  and  executed 

•  IRB  approval  was  sought  and  obtained  for  all  necessary  modifications,  and  continuing 
review  approval  was  obtained. 

•  Recruitment  goals  were  exceeded  with  a  final  randomization  number  of  N  =  420. 

•  All  post-test  and  follow-up  questionnaires  were  completed  in  October  2015. 

•  139  continuing  education  certificates  were  issues. 

•  All  data  collection  was  completed  on  January  7th,  2016. 

•  Data  cleaning,  recording,  and  data  set  preparation  has  begun 

•  Statistical  analysis  plans  were  drafted  for  three  manuscripts,  and  are  in  the  process  of 
finalization.  Prelimnary  analyses  have  been  completed. 

•  The  development  of  three  manuscripts  (baseline,  main  results,  web  usage)  are 
underway. 

During  the  upcoming  performance  period,  year  5,  the  team  will  focus  on  the  following  activities 
and  milestones: 

1 .  Create  final  analytic  data  sets  -  Data  sets  are  being  finalized.This  task  will  continue  into 
Year  5. 

2.  Perform  all  analyses  according  to  specification  -  During  Year  5,  the  team  will  finalize 
SAPs  for  each  proposal  and  perform  statistical  analyses  to  address  study  aims  (above). 

3.  Author  and  co-author  evaluation  findings  -  During  Year  5,  findings  will  be  analyzed  and 
discussed  in  developed  manuscripts.  We  anticipate  submitted  3-4  papers  prior  to  the 
termination  of  the  grant. 

4.  Convene  advisory  meeting  and  project  review  -  During  Year  5,  the  team  will  review 
study  achievements  and  lessons  learned. 

To  date  there  have  been  no  risks  associated  with  this  project  that  have  impeded  its 
performance.  With  the  complex  nature  of  the  data  analysis,  all  data  analysis  and  paper  writing 
has  not  yet  been  completed.  VA  has  submitted  a  request  for  a  first  time  Extension  Without 
Funds  to  extend  the  performance  period  to  September  29th,  2017  in  order  to  complete  all  data 
analysis  and  manuscript  writing. 
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The  need  for  additional  personnel  will  be  reassessed  upon  approval  of  the  extension  without 
funds. 


REPORTABLE  OUTCOMES 

Dr.  Ruzek  presented  an  update  on  the  project  to  CDMRP  leadership  at  the  MOMRP  meeting  in 
Ft.  Detrick  in  September  2016. 

All  presentations  to  date: 


Title 

Presenter 

Conference  Dates 

A  Randomized  Control  Trial  of  Online  Training 
for  PTSD 

Alie,  G.  &  Graham,  B.C. 

April  2013 

Actor  Fidelity  and  Training  for  Standarized 

Patient  use  in  Clinical  Research 

Jordan,  K.R.  &  Graham, 

B.C. 

April  2014 

Web-centered  Consultation  for  Online  Training 
in  PTSD  Treatment:  A  Scalable  Approach  to 
Skill-building 

Sardarian,  S.  &  Graham, 

B.C. 

April  2014 

Individual  and  Organizational  Factors  in 
Dissemination  and  Implementation  of  Skills 
following  an  Online  Training  for  Clinicians 
Treating  PTSD 

Graham,  B.C.,  Ruzek,  J.  & 
Lee,  J.  E. 

May  1  -  May  3,  2014 

Randomized,  Controlled  Trial  of  CBT  Training 
for  PTSD  Providers:  Project  OUTFIT 

Ruzek,  J.  &  Rosen,  R. 

August  5  -  August  6, 

2014 

Project  OUTFIT :  Online  User  T raining  for 
Intervention  in  Trauma 

Graham,  B.C.  &  Ruzek,  J. 

April  28,  2015 

The  Role  of  Organizational  and  Provider 

Factors  in  Community  Interventions  Across 
Diverse  Systems. 

Graham,  B.C. 

June  1,  2015 

Randomized,  Controlled  Trial  of  CBT  Training 
for  PTSD  Providers:  Project  OUTFIT 

R.  Rosen 

September  11, 2015 

Improving  the  Efficiency  of  Standardized 

Patient  Assessment  of  Clinician  Fidelity:  A 
Comparison  of  Automated  Actor-based  and 
Manual  Clinician-based  Ratings 

Graham,  B.C. 

September  25  - 
Septmber  26,  2015 

Evaluating  Clinicians’  Use  of  a  Web-based 
Training  in  Cognitive  Behavioral  Therapy  for 
PTSD:  Impact  on  Outcomes  and  Correlation 
with  Self-Reported  Use 

Coleman,  J. 

August  6,  2016 

Assessment  of  Evidence-Based  Training  for 
Trauma  Providers:  Project  OUTFIT  Measures 
&  Outcomes 

Rosen,  R. 

August  15-18,  2016 

Evidence-Based  Training  of  PTSD  Providers: 
Results  of  a  Randomized  Controlled  Trial  of 
On-Line  Training  for  VA  and  Community-based 
PTSD  Providers 

Rosen,  R.  &  Wilkinson,  A. 

August  15-18,  2016 

ISTSS  Symposium  -  Challenges  in 

Development  of  Scalable,  Effective  Training  for 

R.  Rosen,  J.  Ruzek,  J. 
Coleman 

November  11th,  2016 
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Mental  Health  Providers 


CONCLUSION 

This  training  program  focused  on  the  development  of  evidence-based  CBT  skills  to  improve 
skills  in  providers  treating  Veterans  and  active  duty  military  and  to  effectively  engage  patients  in 
the  treatment  process.  This  innovative  study  will  add  new  knowledge  to  our  understanding  of 
skills  dissemination  in  PTSD  provider  care.  We  tested  the  hypotheses  of  the  study  in  a  rigorous, 
experimental  design,  and  assessed  outcomes  of  new  web-based  training  modules  and 
consultation  methods.  This  study  will  provide  data  to  assist  researchers,  military  leaders,  and 
treatment  providers  to  better  understand  practical  and  theoretical  implications  for  future  training 
of  mental  health  providers  in  the  VHA  and  other  health  systems. 

Data  analysis,  manuscript  writing,  and  final  advisory  board  meeting  will  occur  over  the  next  12 
months  of  grant  activity  (pending  approval). 
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Randomized,  Controlled  Trial  of  CBT  Training  for  PTSD  Providers  Period  of  Performance 
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Primary  and  Secondary  Aims 

•Primary  Aim:  To  compare  an  enhanced,  internet-based  training  intervention  combined  with  novel  web- 
centered  supervision,  internet-based  training  intervention  without  web-centered  supervision  and  a  written 
materials-only  control  with  regard  to  improvements  in  CBT  skills. 

•Secondary  Aim  #1  To  compare  improvements  in  knowledge  and  attitudes  following  internet-based 
training  with  or  without  web-centered  supervision  and  a  written  materials-only  control. 

•Secondary  Aim  #2:  To  compare  improvements  in  skills  acquisition  in  knowledge  and  attitudes  following 
training  in  clinicians  recruited  from  VHA  mental  health  treatment  settings  compared  to  those  providing 
services  in  civilian  community-based  clinics. 

•Secondary  Aim  #3:  To  determine  whether  clinician  implementation  of  skills  assessed  by  means  of  a 
novel,  patient-based  measure  of  clinician  skills  implementation  and  effectiveness  is  predictive  of  changes 
in  an  objective  (i.e.,  standardized  patient)  measure  of  skills. 

•Secondary  Aim  #4:  To  assess  the  relative  efficiency  of  training,  as  measured  by  total  time  required  for 
training  in  each  condition,  in  addition  to  self-reported  level  of  burden  for  clinicians. 


Approach 

These  aims  will  be  accomplished  by  systematically  comparing  web-based  training  interventions  across 
treatment  settings  and  provider  groups  (VHA  vs  Non-VHA).  Other  unique  qualities  of  this  study  include 
developing  and  testing  new  web-enhanced  training  modules  and  a  novel  web-centered  supervision 
model,  the  development  and  implementation  of  a  new  patient-reported  measure  of  clinician  skill  and 
competency  and  lastly,  assessment  of  post-training  maintenance  of  skills  beyond  the  training  period.  This 
study  focuses  on  the  broadly  applicable  CBT  skill  areas  of  Behavioral  Task  Assignment  and  Chain 
Analysis.  If  successful,  the  study  will  promote  a  better  standard  of  care  for  the  psychological  health  of 
Veterans  and  their  families. 


Timeline  and  Cos 


Activities 

CY 

12-13 

CY 

13-14 

CY 

14-5 

CY 

15-6 

Develop  content  and  web  based 
training/Pilot 

1 

Recruitment/Enrollment 

1 

Conduct  training  interventions 
and  outcome  assessments 

m 

Data  analysis,  manuscript 
preparation,  dissemination 

IZl 

Estimated  Total  Budget  ($K) 

322 

1,074 

767 

301 

Updated:  10/25/2016 


Data  cleaning  has  been  completed.  The  team  is  in  the  process  of  formulating  analyses 
plans  for  approximately  four  papers  based  on  the  findings.  Conversion  of  the  web-based 
training  program  from  CMS  to  HTML  has  been  completed. 


Goals/Milestones 

CY12-13  Goal  -  Design  training  modules  and  survey  instruments 
0  PTSD  training  material  completed 
0  Web  modules  programmed 

0  Study  instruments  completed  or  on  time  for  completion 
0  CY13-14  Goals  -  Initiate  data  collection 

0  Begin  recruitment  of  participants  through  established  VA  and  Give  an  Hour  networks 
0  Begin  phone  supervision  in  the  web-centered  supervision  condition 
CY14-15  Goal  - 
0  Continue  data  collection 

0  Assemble  rater  teams  for  standardized  patient  ratings 
0  Complete  recruitment  of  remaining  participants 

CY15-16  Goal 

□  Transfer  of  web-based  modules  to  VHA  servers 

□  Coding,  analysis,  and  syntheses  of  learning  consultation  data.  Writing  of  results  and  dissemination 

□  Analysis  of  baseline  data,  writing  of  results,  and  dissemination 

□  Analysis  of  main  findings,  writing  of  results,  and  dissemination 

□  Disseminate  modules  via  VHA  servers 

Comments/Challenges/lssues/Concerns 

•  We  have  recently  submitted  an  Extension  without  Funds  to  extend  our  end  of 
date  to  29Sept2017 

Budget  Expenditure  to  Date 
Projected  Expenditure:  $2,464,000 
Actual  Expenditure:  $992,645 


